Interactive 3d cursor for use in medical imaging

ABSTRACT

An interactive 3D cursor facilitates selection and manipulation of a three-dimensional volume from a three-dimensional image. The selected volume image may be transparency-adjusted and filtered to remove selected tissues from view. Qualitative and quantitative analysis of tissues in a selected volume may be performed. Location indicators, annotations, and registration markers may be overlaid on selected volume images.

TECHNICAL FIELD

Aspects of this disclosure are generally related to human-machineinterfaces, and more particularly to cursors.

BACKGROUND

The typical arrow-shaped cursor presented by a computer operating systemis zero-dimensional. A zero-dimensional cursor designates the locationof a single point in a space such as a two-dimensional window presentedon a monitor. Mouse buttons can be used in combination with movement ofthe cursor to select objects in the two-dimensional space, but at anygiven instant of time a zero-dimensional cursor position designates onlya single point in space.

The current standard for diagnostic radiologists reviewing computedtomography (CT) or magnetic resonance imaging (MRI) studies is aslice-by-slice method. A conventional keyboard, monitor, and mouse witha zero-dimensional cursor are used for manipulating the images. The useof mouse buttons and cursor movement for manipulating the images canbecome burdensome. For example, many images are included in radiologystudies that are performed for the follow up of cancer to determine theresponse to treatment. The ability to recognize and analyze differencesbetween images can be important. As an example, the recent Investigationof Serial Studies to Predict Your Therapeutic Response with Imaging andMolecular Analysis (I-SPY) trial tracked the changes in the tumor overmultiple magnetic resonance imaging (MM) scans during the administrationof neoadjuvant chemotherapy (NACT). It has been noted that thephenotypic appearance (e.g., shape, margins) of a tumor correlated withthe pathologic response to NACT. A more efficient and accurate interfacefor manipulating and presenting medical images would therefore haveutility.

Known techniques for 3D viewing of medical images are described in U.S.Pat. No. 9,349,183, Method and Apparatus for Three Dimensional Viewingof Images, issued to Douglas, U.S. Pat. No. 8,384,771, Method andApparatus for Three Dimensional Viewing of Images, issued to Douglas,Douglas, D. B., Petricoin, E. F., Liotta L., Wilson, E. D3D augmentedreality imaging system: proof of concept in mammography. Med Devices(Auckl), 2016;9:277-83, Douglas, D. B., Boone, J. M., Petricoin, E.,Liotta, L., Wilson, E. Augmented Reality Imaging System: 3D Viewing of aBreast Cancer. J Nat Sci. 2016;2(9), and Douglas, D. B., Wilke, C. A.,Gibson, J. D., Boone, J. M., Wintermark, M. Augmented Reality: Advancesin Diagnostic Imaging. Multimodal Technologies and Interaction,2017;1(4):29. In D3D imaging, the radiologist wears an augmented reality(AR), mixed reality (MR) or virtual reality (VR) headset and uses ajoystick or gaming controller. Advantages include improved depthperception and human machine interface. Still, there are severalchallenges faced with this approach. First, an area of interest (e.g.tumor) may be in close proximity to structures that are similar incomposition/density. Isolating the area of interest for betterexamination may be difficult. Second, many soft tissues in the body aremobile and deformable, so it can be difficult to achieve the bestorientation to properly compare the tumor at multiple time points.Efficiently aligning the orientation to do so may be difficult. Third,certain portions of a tumor can respond to treatment and decrease insize while other portions of a tumor demonstrate increases in size. Thepattern of tumor shrinkage has important prognostic implications.Furthermore, composition and complex morphologic features includingspiculations (spikes extending from the surface), irregular margins andenhancement also have important implications. Consequently, there is aneed for a system that facilitates recognition of the subtle, yetimportant changes in size, shape and margins. Fourth, a patient withmetastatic cancer has several areas of interest in different areas ofthe body. It is difficult and time consuming to find each of the areasof interest at every time point to determine interval change.Consequently, there is a need for a system that enables the observer todo this efficiently.

SUMMARY

All examples, aspects and features mentioned in this document can becombined in any technically possible way.

In accordance with an aspect of the invention a method comprises:generating a three-dimensional cursor that has a non-zero volume;responsive to a first input, moving the three-dimensional cursor withina three-dimensional image; responsive to a second input, selecting avolume of the three-dimensional image designated by thethree-dimensional cursor; and responsive to a third input, presenting amodified version of the selected volume of the three-dimensional image.In some implementations presenting the modified version of the selectedvolume of the three-dimensional image comprises removing an un-selectedvolume of the three-dimensional image from view. In some implementationspresenting the modified version of the selected volume of thethree-dimensional image comprises changing transparency of presentedtissues within the selected volume. In some implementations presentingthe modified version of the selected volume of the three-dimensionalimage comprises filtering a selected tissue to remove the selectedtissue from view. In some implementations presenting thethree-dimensional cursor with measurement markings on at least one edge,surface or side. In some implementations presenting the modified versionof the selected volume of the three-dimensional image comprisespresenting inputted location indicators. In some implementationspresenting the modified version of the selected volume of thethree-dimensional image comprises presenting inputted annotations. Someimplementations comprise changing a size dimension of thethree-dimensional cursor responsive to a fourth input. Someimplementations comprise changing a geometric shape of thethree-dimensional cursor responsive to a fifth input. Someimplementations comprise automatically generating a statisticalrepresentation of the selected volume of the three-dimensional image. Insome implementations presenting the modified version of the selectedvolume of the three-dimensional image comprises presenting at least onetissue type with false color. In some implementations presenting themodified version of the selected volume of the three-dimensional imagecomprises presenting volumetric changes over time with false color. Someimplementations comprise presenting multiple computed tomography imagesassociated with the selected volume using reference lines. Someimplementations comprise presenting multiple axial computed tomographyimages associated with the selected volume using reference lines. Someimplementations comprise presenting a maximum intensity projection (MIP)image of a positron emission tomography (PET) scan with thethree-dimensional cursor overlaid thereon to indicate orientation andlocation of the selected volume. Some implementations comprisepresenting a radiology report enhanced with information obtained usingthe three-dimensional cursor. Some implementations compriseautomatically calculating and presenting a quantitative analysis and aqualitative analysis associated with multiple time points. Someimplementations comprise presenting the modified version of the selectedvolume of the three-dimensional image comprises presenting inputtedregistration markers. Some implementations comprise automaticallycalculating volumetric change based on the registration markers. Someimplementations comprise automatically re-orienting the selected volumeof the three-dimensional image based on the registration markers. Someimplementations comprise using multiple volumes selected with thethree-dimensional cursor to designate a pre-operative planning pathwayfor guiding surgical intervention. Some implementations comprisepresenting the selected volume with an augmented reality, virtualreality or mixed reality headset.

In accordance with an aspect of the invention an apparatus comprises: acomputing device; and a human-machine interface comprising athree-dimensional cursor that has a non-zero volume; the human-machineinterface moving the three-dimensional cursor within a three-dimensionalimage responsive to a first input; the human-machine interface selectinga volume of the three-dimensional image designated by thethree-dimensional cursor responsive to a second input; and thehuman-machine interface presenting a modified version of the selectedvolume of the three-dimensional image responsive to a third input. Insome implementations, the human-machine interface removes an un-selectedvolume of the three-dimensional image from view. In someimplementations, the human-machine interface changes transparency ofpresented tissues within the selected volume. In some implementations,the human-machine interface filters a selected tissue to remove theselected tissue from view. In some implementations, the human-machineinterface presents the three-dimensional cursor with measurementmarkings on at least one edge, surface or side. In some implementations,the human-machine interface receives and implements inputted locationindicators. In some implementations, the human-machine interfacereceives and implements inputted annotations. In some implementations,the human-machine interface changes a size dimension of thethree-dimensional cursor responsive to a fourth input. In someimplementations, the human-machine interface changes a geometric shapeof the three-dimensional cursor responsive to a fifth input. In someimplementations, the human-machine interface automatically generates andpresents a statistical representation of the selected volume of thethree-dimensional image. In some implementations, the human-machineinterface presents at least one tissue type with false color. In someimplementations, the human-machine interface presents volumetric changesover time with false color. In some implementations, the human-machineinterface presents multiple computed tomography images associated withthe selected volume using reference lines. In some implementations, thehuman-machine interface presents multiple axial computed tomographyimages associated with the selected volume using reference lines. Insome implementations, the human-machine interface presents a maximumintensity projection (MIP) image of a positron emission tomography (PET)scan with the three-dimensional cursor overlaid thereon to indicateorientation and location of the selected volume. In someimplementations, the human-machine interface presents a radiology reportenhanced with information obtained using the three-dimensional cursor.In some implementations, the human-machine interface automaticallycalculates and presents a quantitative analysis and a qualitativeanalysis associated with multiple time points. In some implementations,the human-machine interface presents inputted registration markers. Insome implementations, the human-machine interface automaticallycalculates volumetric change after appropriate registration using theregistration markers. In some implementations, the human-machineinterface automatically re-orients the selected volume of thethree-dimensional image based on the registration markers. In someimplementations, the human-machine interface presents multiple volumesselected with the three-dimensional cursor to designate a pre-operativeplanning pathway for guiding surgical intervention. In someimplementations, the human-machine interface presents the selectedvolume with an augmented reality, virtual reality or mixed realityheadset.

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1A illustrates a 3D cursor selecting a volume of interest from athree-dimensional medical image.

FIG. 1B illustrates the volume of interest selected with the 3D cursor;

unselected portions have been removed from view.

FIG. 1C illustrates modification of the transparency of the selectedvolume of interest.

FIG. 1D illustrates filtering of selected areas of the selected volumeof interest.

FIG. 2 illustrates a variant of the 3D cursor of FIG. 1A withmeasurement markings on edges and sides.

FIG. 3 illustrates location indicators and annotations positionedrelative to the portion of the image within the selected volume ofinterest.

FIGS. 4A, 4B, and 4C illustrate three different examples of geometricshapes of the 3D cursor of FIG. 1A.

FIG. 5 illustrates presentation of a quantitative analysis of tissuesinside of the volume of interest selected with the 3D cursor of FIG. 1A.

FIG. 6 illustrates use of false color and transparency changes toenhance viewing of the selected volume of interest.

FIG. 7 illustrates association of multiple computed tomography (CT)images of the chest in lung windows with the interactive 3D cursor usingreference lines.

FIG. 8 illustrates association of multiple axial computed tomography(CT) slices of the chest in lung windows with the interactive 3D cursorusing reference lines.

FIG. 9 illustrates a maximum intensity projection (MIP) image of afludeoxyglucose (18F) positron emission tomography (PET) scan in whichtwo varying sized interactive 3D cursors are overlaid to indicate 3Dcursor shape, size, orientation, and location when respective volumes ofinterest were selected.

FIG. 10 illustrates a radiology report enhanced with informationobtained using the interactive 3D cursor and including quantitative andqualitative analysis.

FIG. 11 illustrates a radiology report enhanced with informationobtained using the interactive 3D cursor, and including addedquantitative and qualitative analysis at multiple time points.

FIGS. 12A, 12B and 12C illustrate a technique for correction formis-registration at multiple time points using three or more markers.

FIG. 13 illustrates use of multiple interactive 3D cursors to selectvolumes of interest to designate a safe pre-operative planning pathwayfor guiding surgical intervention.

FIG. 14 illustrates use of the interactive 3D cursor in an educationalsetting.

FIG. 15 illustrates process steps on a radiologist's review of apatient's image with integration of the interactive 3D cursor.

FIG. 16 illustrates a system for use of the interactive 3D cursor.

DETAILED DESCRIPTION

Some aspects, features and implementations described herein may includemachines such as computers, electronic components, radiologicalcomponents, optical components, and processes such ascomputer-implemented steps. It will be apparent to those of ordinaryskill in the art that the computer-implemented steps may be stored ascomputer-executable instructions on a non-transitory computer-readablemedium. Furthermore, it will be understood by those of ordinary skill inthe art that the computer-executable instructions may be executed on avariety of tangible processor devices. For ease of exposition, not everystep, device or component that may be part of a computer or data storagesystem is described herein. Those of ordinary skill in the art willrecognize such steps, devices and components in view of the teachings ofthe present disclosure and the knowledge generally available to those ofordinary skill in the art. The corresponding machines and processes aretherefore enabled and within the scope of the disclosure.

FIG. 1A illustrates a 3D (three-dimensional) cursor 100 overlaid on athree-dimensional medical image 102. In the illustrated example, the 3Dcursor 100 defines a cubic volume of interest. The medical image 102could include any portion of a body, or an entire body, for example andwithout limitation. For purposes of explanation the medical image 102includes different types of tissue. More specifically, the imageincludes a background material 104, such as fat, a lobulated mass 106, atubular-shaped vein 108, and an artery 110. The 3D cursor 100 can bemoved relative to the image, e.g. in three dimensions, such as bymanipulating an IO device such as a 3D mouse, for example and withoutlimitation. A button click or other input designates (selects) theportion of the image that is located inside the three-dimensional volumeof the 3D cursor 100. Distinguishing between a 3D image portion selectedby a 3D cursor and other unselected image portions is described in US2016/0026266 and U.S. Pat. No. 8,384,771, both of which are incorporatedby reference.

FIG. 1B illustrates the selected image portion of FIG. 1A. Moreparticularly, unselected portions of the image located outside of animage portion 112 selected with the 3D cursor 100 have been filtered-outor otherwise completely removed from view. Consequently, the removedportions of the image do not obstruct or hinder the view of the selectedimage portion. Moreover, the selected image portion 112 can bemanipulated and viewed as a separate and distinct image from the largermedical image 102 from which it was selected.

FIG. 1C illustrates modification of the transparency of the selectedimage portion 112. More specifically, transparency may be decreasedand/or increased such that tissues and other features can be betterobserved, e.g. such that overlapping tissues and features are visible.For example, tissues and features located proximate to the back of theselected image portion such as lobulated mass 106 can be seen throughoverlapping tissues and features located proximate to the front of theselected image portion such as vein 108, when transparency issufficiently increased. The transparency may be manipulated with the IOdevice to achieve various levels of transparency. Further, differentlevels of transparency may be applied to different portions of theselected image portion.

FIG. 1D illustrates filtering of selected areas or tissues of theselected image portion 112 to remove those areas or tissues from view.In the illustrated example the background material 104, vein 108, and anartery 110 have been removed from view, leaving only the lobulated mass106. The tissues to be filtered (removed from view) may be selectedbased on geometric shape, color, brightness, density, and any other of avariety of available image data, either alone or in combination.Moreover, a designated volume defined by a geometric shape may beremoved, e.g. a geometric shape that traverses tissue boundaries.

Transparency modification and tissue filtering facilitate presentationof certain tissue types of concern, both within the cursor and outsideof the cursor. Currently, the medical professional must see through anytissue within the cursor but external to the tissue type of concern fromthe viewing point of the medical professional, thus degrading thevisibility of the tissue of concern. The illustrated improvements enablethe medical professional to change the transparency of any tissue withinthe cursor-defined volume but external to the tissue type of concern.Alternatively, tissue types not of concern are subtracted from thevolume contained within the interactive 3D cursor, leaving only thetissue of concern in the presented image. Multiple interactive 3Dcursors in combination can be used to obtain varying patterns of tissuesubtraction. This helps to overcome the limitations of degradedvisibility due to tissue within the cursor but external to the tissuetype of concern from the viewing point of the medical professional.

FIG. 2 illustrates an implementation of the 3D cursor 100 withdimensional measurement markings. Dimensional measurement markings maybe available as a feature that can be turned ON and OFF. In theillustrated example, the 3D cursor is a 2 cm by 2 cm by 2 cm cube. Thedimensional measurement markings include tick marks 200, 202, and 204that respectively designate 1 mm, 5 mm, and 1 cm increments along theedges of the cube (and thus representing three dimensions). Tick marksthat represent different magnitudes may be uniquely represented tofacilitate visual size determination of the lobulated mass 106 thatrepresents the lesion of interest. 1 cm markings 206 are presented ineach of two dimensions on each side of the cube.

The dimensional measurement markings can help serve as a reference forradiologist's activities to include visual assessment, orientation,comparisons with prior scans or measurements. Advantages may includemitigating the current lack of metrics are available to the medicalprofessional to understand the size of the cursor and/or of the tissueelements contained within the cursor. This implementation placesmeasurement metrics on each edge and side of the cursor to help enablethe medical professional to rapidly understand the size of the subtendedvolume within the cursor. In the case where the cursor encapsulates avolume of concern such as a tumor, the three-dimensional size could berecorded in the medical professional report. This can help the visualassessment of each portion of the tumor to aid in the assessment ofsmall changes in size of findings including lobulations of a mass'smargin and spiculations.

Referring to FIG. 3, location indicators 300 and annotations 302 may beplaced by the radiologist or by automated techniques to highlightlocations or regions of concern within the interactive 3D cursor. Thelocation indicators may specify a point or region within the volume ofthe 3D cursor. Annotations can be added manually by the radiologist orby automated techniques to describe areas that are of concern, e.g.,growing, spiculation, irregular margin, indistinct margin, etc. Ifspiculations are on the surface of a tumor, this could be an indicatorof potential malignancy. The location indicators, such as, but notlimited to, arrow(s) pointing to key regions of interest within/outsidethe 3D cursor helps to overcome the limitation of the inability to markkey points within the cursor. This feature will be useful in discussionsbetween medical professions regarding a patient's condition. It willalso be useful in communicating imaging findings between a medicalprofessional and a patient.

Referring to FIGS. 4A, 4B, and 4C, the 3D cursor may be may beimplemented in a wide variety of different shapes. Examples include butare not limited to cube, cuboid, cylinder, sphere, ellipsoid, cone andtetrahedron. The shapes are not necessarily regular, and the lengths ofedges may be resized, e.g. overall geometric shape scaling or changingindividual edges, sides, or surfaces. For example, FIGS. 4A and 4Billustrate cuboid 3D cursors 400, 402 for which edge length has been setor selected based on the dimensions and orientation of the respectivefeature of interest 404, 406. FIG. 4C illustrates a spherical 3D cursor408 for which the diameter may be set or selected based on thedimensions of the feature of interest 410. In addition to dimensionalchanges, cursor geometric shape may be changed.

The ability to change the size, shape, and individual dimensions of the3D cursor enables the cursor to be customized based on the particularvolume of interest to the medical professional. A fixed-shape,fixed-size cursor might be too large or too small, e.g. so as to includea significant amount of tissue not of interest. For example, inexamining the lungs, placement of a cube-shaped cursor could cause ribsto be included in the image. Changing the shape of the 3D cursor wouldhelp to overcome this limitation. Customization could be accomplished bywide variety of techniques, possibly including but not limited toselecting an edge, side or vertex of the original 3D cursor with asecond type of cursor 412, and then “clicking and dragging” the selectededge, side, or vertex in the desired direction to expand or reduce thevolume of the original 3D cursor. The interface may also enableselection and change between multiple 3D geometric shapes, e.g. changingfrom cuboid to spherical. Scrolling on the conventional slices whilesimultaneously drawing shapes can also be performed to generate theprescribed 3D cursor volume. The interactive 3D cursor thus provides anefficient interface for tissue subtraction to provide enhancedvisualization of the tumor.

FIG. 5 illustrates presentation of a quantitative analysis 500 of alltissues inside a volume selected with the 3D cursor. The illustratedexample includes a bar graph but it is to be understood that any of awide variety of charts, graphs, and other techniques for presentation ofdata might be implemented. Quantitative analysis can help theradiologist understand how a feature of interest such as tumor 502(e.g., the lobulated mass 106, FIG. 1B) is changing in volume 504 overmultiple time points. The interface may include a statisticalrepresentation of the tissue types, possibly including but not limitedto a histogram bar chart to depict the volume (e.g., number of voxelsper unit volume) of the different types of tissue within the cursor,distinct markings for different types of tissue such as, but not limitedto, color coding the bars of the histogram bar chart.

FIG. 6 illustrates an implementation of the interactive 3D cursor 100with false color and transparency to enhance viewing. False color andtransparency may be dynamically adjusted and turned ON and OFF.Different false colors may be applied to different tissue types withinthe volume of the 3D cursor. The colors could be selected to correspondto the colors used in the statistical representation (FIG. 5).Alternatively, a respective unique false color could be selected foreach different tissue type, or tissue types of particular interest orconcern, and/or additional features of concern, e.g., irregular margin,indistinct margin, spiculation, etc. In the illustrated example, thebackground material 104 (fat) is depicted in light gray, the artery 110is depicted in red, the vein 108 is depicted in blue, and the lobulatedmass 106 is multicolored. Different colors may be selected or used toindicate stability of the lobulated mass 106 over time. For example,green may be used to indicate a stable volume 112 while orange is usedto denote a slow growth volume 114, thereby providing a visual warningindicator. Red may be used to indicate high rate of growth or concerningmargin volume 116. The extent of the volume of the lobulated mass can bedetermined automatically, e.g. based on density. Moreover, changes involume of sub-regions of the lobulated mass may also be automaticallydetermined, and color coding may be automatically implemented. This canhelp the radiologist understand how the mass is changing in volume overmultiple time points.

FIG. 7 illustrates association of multiple computed tomography (CT)images of the chest in lung windows with the interactive 3D cursor 100using reference lines 700. The illustrated example includes an axialimage 702, a sagittal image 704, and a coronal image 706 of the chest inlung windows. An advantage is enhanced ability to cross reference the 3Dcursor to the original 2D slices 702, 704, 706 from which total 3Dvolume was obtained. Medical professionals have experience andfamiliarity with 2D slices and may feel more confident in their findingsgiven the capability to switch back and forth between the 2D and 3Dvolumetric approaches. A small display adjacent to the interactive 3Dcursor could indicate which 2D slices contain tissue within in theinteractive 3D cursor. Then the medical professional could direct thesystem to automatically select those slices which have tissue within thecursor and display them on a nearby 2D display unit. A correspondingvisible boundary of the 3D cursor (e.g., red) projected on each of theslices may be presented.

FIG. 8 illustrates association of multiple axial computed tomography(CT) slices 800, 802, 804, 806 of the chest in lung windows with theinteractive 3D cursor 100 using reference lines 808. The multiple axialcomputed tomography (CT) slices of the chest in lung windows show thelocation of the 3D cursor, i.e. the slice area that includes across-section of the 3D cursor, which in the illustrated example hasselected a left upper lobe mass. Boundaries 810 of the 3D cursor in theslices are depicted in a color, e.g. red. Within the 3D cursor the lungcancer mass 106 is depicted in gray, surrounded by black that indicatesnon-cancerous lung tissue. This implementation helps the medicalprofessional to rapidly visualize where the interactive 3D cursor islocated relative to the slice images and the body. It also enables themedical professional to visualize the entire volumetric data with theinteractive 3D cursor accurately positioned within the volume.Transparency of tissue within the 3D volume could be changed so that theinteractive 3D cursor would stand out. This would help avoid left—rightorientation mistakes that might occur during treatment. Multipleinteractive 3D cursors which could be of differing sizes and/or shapescould be created and displayed.

FIG. 9 illustrates overlay of 3D cursors 100 a, 100 b on a maximumintensity projection (MIP) image 900 of a fludeoxyglucose (18F) positronemission tomography (PET) scan. Two different-sized interactive 3Dcursors are used to highlight two separate areas of concern, including3D cursor 100 a for a right lung mass and cursor 100 b for a vertebralbody metastasis. This helps to automatically transfer data (e.g.,picture of tissue within the cursor and statistical representations)from the viewing modality to the report of findings. Selection of keydata through human machine interface such as, but limited to, a screencapture can be automatically transferred to the report of findings. Thiswould provide quantitative results within the report together withqualitative impressions of the medical professional.

FIG. 10 illustrates a radiology report 1000 enhanced with informationobtained from the interactive 3D cursor. Qualitative findings 1002 andquantitative findings 1004 may be included along with patientidentifying information 1006, clinical history 1008, comparisons 1010,conclusions 1012, and recommendations 1014. Also included are a selectedvolume image 1016 and statistical graphic 1018. This helps toquantitatively track changes in volumes of concern (e.g., tumors) overtime.

FIG. 11 illustrates a radiology report 1100 enhanced with informationobtained from the interactive 3D cursor at multiple time points.Qualitative findings 1002 and quantitative findings 1004 may be includedalong with patient identifying information 1006, clinical history 1008,comparisons 1010, conclusions 1012, and recommendations 1014. Alsoincluded are selected volume images 1102, 1104 from different timepoints and respective statistical graphics 1106, 1108 from those timepoints. Follow up reports can include current and prior exams 1110, 1112with quantitative analysis and qualitative analysis on how the lesionhas changed over time. This may facilitate selection of a lesion (e.g.,tumor) at multiple time points using an interactive 3D cursor;qualitative assessment of the lesion at multiple time points; and,quantitative assessment of the lesion at multiple time points. Thiswould enable the medical professional to better assess how a particularlesion is changing over time. A report of current findings as outlinedin the previous implementation could be placed in a report together withthe data obtained from an earlier examination. This would enabletracking over time the progress of treatment or that of changes intissues of interest/concern.

FIGS. 12A, 12B, and 12C illustrate a registration technique by whichmis-registration can be corrected at multiple time points through theuse of three or more markers 12, 14, 16. Initially, the mass 106 withineach 3D cursor 100 image is noted using different locations within theinteractive 3D cursor and different orientations. Next, the user markssimilar locations on each image of the mass with registration markers.In the illustrated example, a yellow marker 12, a red marker 14, and ablue marker 16 correspond to the same respective parts of the mass oneach scan. Finally, tissues within the interactive 3D cursor are alignedin accordance with markers. Many soft tissues within the body can changein orientation from one scan to the next due to patient movement.Corresponding mis-registration can limit the ability to properly trackhow a lesion changes over time. This technique provides a method tocorrect for such mis-registration. Three or more recognizable spots ofthe lesion (e.g., tumor) can be marked with a false color, arrow, orother registration mark. Then, these locations can be automaticallyaligned with one another. Shadows can be added to help bring out depthperception. Proper alignment will accurately align the shadows. Thisenhances visual assessment for how a lesion is changing over time toinclude changes in tumor composition, size and morphology.

FIG. 13 illustrates use of multiple image volumes selected with the 3Dcursor to designate a safe pre-operative planning pathway to guidesurgical intervention. In the illustrated example, multiple greeninteractive 3D cursors 1300 mark a surgeon-selected dissection pathwaythat is deemed safe in the pre-operative setting. The interactive 3Dcursor 100 containing the cancerous lesion 106 is shown at a distal endof the planned surgical path represented by abutting or overlappingvolumes selected with the 3D cursors 1300. The selected path that thesurgeon will excise avoids the artery 110 with a minimum clearance of 10mm. This provides the advantage of 3D depiction of possible surgicalcuts. The path could include, but is not limited to, one or more of thefollowing properties: a serpentine shape; measurements couldsubsequently be made to measure absolute distance between a point on theplanned path to some region of concern (e.g., artery); the path couldalso be projected on a head mounted display at different intervalsduring the course of the operation. This feature would facilitatesurgical planning as well as a potential to improve accuracy of thesurgery.

FIG. 14 illustrates use of the interactive 3D cursor in an educationalsetting. Students 1400 are depicted wearing AR (augmented reality)headsets 1402 and an instructor 1404 is pointing to an abnormality onthe board 1406. This facilitates presentation of medical information(e.g., anatomy) in a classroom environment. The interactive 3D cursorcould be placed around the organ of interest and other parts of the bodycould be eliminated. Items from implementations discussed above such asmetrics and arrows could be used. The students would be provided 3D headdisplays and joined into a display system so that they could see thetissue within the interactive 3D cursor. This would eliminate anyconfusion on the part of the students as to what specific detail in theimagery was being discussed.

FIG. 15 illustrates process steps on a radiologist's review of apatient's image with integration of the interactive 3D cursor intohis/her practice. Step 1 is to create an interactive 3D cursor volumeand shape that approximates the size and shape of patient organ/tissuecorresponding to the item currently being inspected on the checklist.Step 2 is to position the interactive 3D cursor over the organ/tissue tobe inspected. The interactive 3D cursor as it is located within thetotal 3D image volume may be presented on a display. Step 3 is tosubtract from view all tissue external to the interactive 3D cursor. Theinteractive 3D cursor may be rotated to permit viewing from multipleangles. If interactive cursors are used at multiple time points to trackhow a particular lesion (e.g., tumor) changes over time, the 3D cursorscan be rotated in synchrony with on another. Step 4 is to generate astatistical representation e.g., a histogram of tissue densities—colorcoded with the types of tissue that are suspicious. Step 5 is tosubtract from view additional tissue within the interactive 3D cursor asdeemed appropriate by the medical professional. Step 6 is to inspect thevolume within the cursor and identify region(s) of interest and placeindicators, annotations, and registration markers relative to region(s)of interest. Step 7 is to extract a statistical representation andcapture imagery showing indicators, annotations, and registrationmarkers and residual tissue within the interactive 3D cursor to beinserted into the medical professional's report. Step 8 is to usecross-referencing as described the above to confirm findings. Step 9 isto iterate on the other items on the checklist until finished. Step 10is to prepare the report of the medical professional's findings. Thisprocedure provides an opportunity to enhance medical image reviewprocess by medical professionals.

FIG. 16 illustrates a system for use of the interactive 3D cursor. Amedical imaging device 1600 is connected to a computer workstation 1602.A wide variety of medical imaging devices and computer workstationscould be used. Images are captured by the medical imaging device andsent to the computer workstation. The computer workstation includesnon-volatile storage, computer-readable memory, processors, and avariety of other resources including but not limited to IO devices thatprovide a human-machine interface. In the illustrated example, the IOdevices include a monitor 1604, keyboard 1606, 3D mouse 1608, and VRheadset 1610. The IO devices are used to prompt a software program thatruns on the computer workstation to perform the various process stepsand implement the various features that have already been describedabove.

There are multiple potential advantages of the interactive 3D cursor.For example, there is reduction in time spent for classification ofmultiple lesions. The radiologist doesn't have to sort through manyprior imaging studies to find the lesion and the interactive 3D cursorwill save time. There is reduction in error when tracking multiplelesions, i.e. reducing the likelihood of mistakes when identifyingdifferent specific lesions that are nearby one another when comparingmultiple scans. One possibility is to analyze the images obtained usingthe 3D cursor and using multiple uniquely tagged (e.g. numbered) cursorsfor any suspicious regions. The medical profession could then switch toslices for confirmation.

Several features, aspects, embodiments and implementations have beendescribed. Nevertheless, it will be understood that a wide variety ofmodifications and combinations may be made without departing from thescope of the inventive concepts described herein. Accordingly, thosemodifications and combinations are within the scope of the followingclaims.

What is claimed is:
 1. A method comprising generating athree-dimensional cursor that has a non-zero volume; responsive to afirst input, moving the three-dimensional cursor within athree-dimensional image; responsive to a second input, selecting avolume of the three-dimensional image designated by thethree-dimensional cursor; and responsive to a third input, presenting amodified version of the selected volume of the three-dimensional image.2. The method of claim 1 wherein presenting the modified version of theselected volume of the three-dimensional image comprises removing anun-selected volume of the three-dimensional image from view.
 3. Themethod of claim 1 wherein presenting the modified version of theselected volume of the three-dimensional image comprises changingtransparency of presented tissues within or external to the selectedvolume.
 4. The method of claim 1 wherein presenting the modified versionof the selected volume of the three-dimensional image comprisesfiltering a selected tissue to remove the selected tissue from view. 5.The method of claim 1 comprising presenting the three-dimensional cursorwith measurement markings on at least one edge, surface or side.
 6. Themethod of claim 1 wherein presenting the modified version of theselected volume of the three-dimensional image comprises presentinginputted location indicators.
 7. The method of claim 1 whereinpresenting the modified version of the selected volume of thethree-dimensional image comprises presenting inputted annotations. 8.The method of claim 1 comprising changing a size dimension of thethree-dimensional cursor responsive to a fourth input.
 9. The method ofclaim 8 comprising changing a geometric shape of the three-dimensionalcursor responsive to a fifth input.
 10. The method of claim 1 comprisingautomatically generating a statistical representation of the selectedvolume of the three-dimensional image.
 11. The method of claim 1 whereinpresenting the modified version of the selected volume of thethree-dimensional image comprises presenting at least one tissue typewith false color.
 12. The method of claim 1 wherein presenting themodified version of the selected volume of the three-dimensional imagecomprises presenting volumetric changes over time with false color. 13.The method of claim 1 comprising presenting multiple computed tomographyimages associated with the selected volume using reference lines. 14.The method of claim 1 comprising presenting multiple axial computedtomography images associated with the selected volume using referencelines.
 15. The method of claim 1 comprising presenting a maximumintensity projection (MIP) image of a positron emission tomography (PET)scan with the three-dimensional cursor overlaid thereon to indicateorientation and location of the selected volume.
 16. The method of claim1 comprising presenting a radiology report enhanced with informationobtained using the three-dimensional cursor.
 17. The method of claim 16comprising automatically calculating and presenting a quantitativeanalysis and a qualitative analysis associated with multiple timepoints.
 18. The method of claim 1 wherein presenting the modifiedversion of the selected volume of the three-dimensional image comprisespresenting inputted registration markers.
 19. The method of claim 18comprising automatically calculating volumetric change based on theregistration markers.
 20. The method of claim 18 comprisingautomatically re-orienting the selected volume of the three-dimensionalimage based on the registration markers.
 21. The method of claim 1comprising using multiple volumes selected with the three-dimensionalcursor to designate a pre-operative planning pathway for guidingsurgical intervention.
 22. The method of claim 1 comprising presentingthe selected volume with an augmented reality, virtual reality or mixedreality headset.
 23. An apparatus comprising a computing device; and ahuman-machine interface comprising a three-dimensional cursor that has anon-zero volume; the human-machine interface moving thethree-dimensional cursor within a three-dimensional image responsive toa first input; the human-machine interface selecting a volume of thethree-dimensional image designated by the three-dimensional cursorresponsive to a second input; and the human-machine interface presentinga modified version of the selected volume of the three-dimensional imageresponsive to a third input.
 24. The apparatus of claim 23 wherein thehuman-machine interface removes an un-selected volume of thethree-dimensional image from view.
 25. The apparatus of claim 23 whereinthe human-machine interface changes transparency of presented tissueswithin the selected volume.
 26. The apparatus of claim 23 wherein thehuman-machine interface filters a selected tissue to remove the selectedtissue from view.
 27. The apparatus of claim 23 wherein thehuman-machine interface presents the three-dimensional cursor withmeasurement markings on at least one edge, surface or side.
 28. Theapparatus of claim 23 wherein the human-machine interface receives andimplements inputted location indicators.
 29. The apparatus of claim 23wherein the human-machine interface receives and implements inputtedannotations.
 30. The apparatus of claim 23 wherein the human-machineinterface changes a size dimension of the three-dimensional cursorresponsive to a fourth input.
 31. The apparatus of claim 30 wherein thehuman-machine interface changes a geometric shape of thethree-dimensional cursor responsive to a fifth input.
 32. The apparatusof claim 23 wherein the human-machine interface automatically generatesand presents a statistical representation of the selected volume of thethree-dimensional image.
 33. The apparatus of claim 23 wherein thehuman-machine interface presents at least one tissue type with falsecolor.
 34. The apparatus of claim 23 wherein the human-machine interfacepresents volumetric changes over time with false color.
 35. Theapparatus of claim 23 wherein the human-machine interface presentsmultiple computed tomography images associated with the selected volumeusing reference lines.
 36. The apparatus of claim 23 wherein thehuman-machine interface presents multiple axial computed tomographyimages associated with the selected volume using reference lines. 37.The apparatus of claim 23 wherein the human-machine interface presents amaximum intensity projection (MIP) image of a positron emissiontomography (PET) scan with the three-dimensional cursor overlaid thereonto indicate orientation and location of the selected volume.
 38. Theapparatus of claim 23 wherein the human-machine interface presents aradiology report enhanced with information obtained using thethree-dimensional cursor.
 39. The apparatus of claim 23 wherein thehuman-machine interface automatically calculates and presents aquantitative analysis and a qualitative analysis associated withmultiple time points.
 40. The apparatus of claim 23 wherein thehuman-machine interface presents inputted registration markers.
 41. Theapparatus of claim 40 wherein the human-machine interface automaticallycalculates volumetric change based on the registration markers.
 42. Theapparatus of claim 40 wherein the human-machine interface automaticallyre-orients the selected volume of the three-dimensional image based onthe registration markers.
 43. The apparatus of claim 23 wherein thehuman-machine interface presents multiple volumes selected with thethree-dimensional cursor to designate a pre-operative planning pathwayfor guiding surgical intervention.
 44. The apparatus of claim 23 whereinthe human-machine interface presents the selected volume with anaugmented reality, virtual reality or mixed reality headset.